Search AI Products and News

In the field of multimodal artificial intelligence (AI), engineers from Apple have collaborated with a research team from Sorbonne University in France on a significant study. Recently, tech media outlet marktechpost published a blog post discussing the application and prospects of early and late fusion models in multimodal AI. The research indicates that early fusion models trained from scratch offer superior computational efficiency and scalability. Multimodal AI aims to process multiple data types simultaneously, such as images and text; however, integrating these diverse sources presents challenges.

In the field of artificial intelligence, the DeepSeek team recently released their latest research成果, introducing an innovative sparse attention mechanism named NSA (Native Sparse Attention). The core objective of this technology is to enhance the speed of long context training and inference, particularly optimized for modern hardware, significantly improving the efficiency of training and inference. The launch of NSA technology brings significant changes to the training of artificial intelligence models. Firstly, it utilizes a series of advancements tailored for modern computational hardware.

ByteDance officially launches its latest Doubao large model 1.5 Pro (Doubao-1.5-pro), which demonstrates outstanding comprehensive capabilities in various fields, successfully surpassing the well-known GPT-4o and Claude3.5Sonnet in the industry. The release of this model marks an important step forward for ByteDance in the field of artificial intelligence. Doubao 1.5 Pro adopts a novel sparse MoE (Mixture of Experts) architecture, utilizing a smaller set of activation parameters for pre-training. This design's innovation...

Recently, Meta has announced a groundbreaking research achievement by developing a new type of memory layer technology that can significantly enhance the factual accuracy of large language models (LLMs) and achieve unprecedented expansion in parameter scale. This technology not only challenges traditional neural network expansion methods but also provides new directions for future AI architecture design. The core of this research lies in utilizing a trainable key-value lookup mechanism to add extra parameters to the model without increasing computational load (FLOPs). This method's core thinking is...

Is AI actually starting to ‘get smarter’?! The latest research from MIT indicates that the internal structure of large language models (LLMs) astonishingly resembles that of the human brain! This study utilized sparse autoencoder techniques to conduct an in-depth analysis of the activation space of LLMs, revealing three hierarchical structural features that are truly remarkable: first, at the microscopic level, researchers discovered the existence of structures akin to ‘crystals’. The surfaces of these ‘crystals’ are composed of parallelograms or trapezoids, similar to familiar lexical analogies such as ‘man:’

The research team has recently introduced an exciting new method called T-FREE, which significantly boosts the operational efficiency of large language models. Scientists from Aleph Alpha, Darmstadt University of Applied Sciences, hessian.AI, and the German Research Center for Artificial Intelligence (DFKI) have jointly released this amazing technology, which is officially named 'Sparse Representation for Memory-Efficient Embedding Without Tokenizers'. Traditionally, we use tokenizers to convert text into a numerical format that computers can understand, but

The Allen Institute for Artificial Intelligence (AI2) has released an open-source large language model OLMoE, aimed at providing high performance and low-cost solutions. This model employs a Sparse Mixture of Experts (MoE) architecture with 7 billion parameters, but through an intelligent routing mechanism, only 1 billion parameters are used per input token, achieving efficient computation. OLMoE includes both general and instruction-tuned versions, supporting a context window of 4096 tokens. Its training data is sourced from a wide range, including Common Crawl, Dolma CC, and Wikipedia.

In the realm of artificial intelligence, Large Language Models (LLMs) are renowned for their exceptional natural language processing capabilities. However, deploying these models in practical applications faces significant challenges due to their high computational costs and memory footprint during the inference phase. To address this issue, researchers have been exploring ways to enhance the efficiency of LLMs. Recently, a method called Q-Sparse has garnered widespread attention.Q-Sparse is a s